【RocketMQ】详解如何搭建一个多主多从的RocketMQ集群 |
您所在的位置：网站首页 › rocketmq 集群配置 › 【RocketMQ】详解如何搭建一个多主多从的RocketMQ集群 |
【RocketMQ】详解如何搭建一个多主多从的RocketMQ集群
|
下载二进制包
下载链接：RocketMQ官网 
注：二进制包是编译好的，拿来可以直接用，如果不需要定制化，改动源码，可选择二进制包 机器准备准备三台机器，每台机器需要安装好jdk，1.8以上版本； 集群使用交叉部署：三台机器，每台各自装一个nameserver，三个broker：一个master，两个其他master的slave； 目录结构： 机器1：namesever-1、master-a、b-slave1、c-slave1 机器2：namesever-2、master-b、a-slave1、c-slave2 机器3：namesever-3、master-c、a-slave2、b-slave2 注意：将二进制包解压后的文件，复制到以上12个文件夹中，使用独立的二进制包进行部署，因为需要将各自的日志文件、数据文件单独存储，如果不需要分离，则可使用同一个包 部署nameserver创建两个目录：data、logs，用于存放数据和日志，rocketmq471是提前解压好的二进制文件目录： RocketMQ主目录：/neworiental/rocketmq01/rocketmq-nameserver/rocketmq471 新建nameserver配置文件：/neworiental/rocketmq01/rocketmq-nameserver/rocketmq471/conf/nameserver.conf 修改nameserver.conf文件： #数据保存目录,需要配置,不配置的话默认在/root/namesrv kvConfigPath=/neworiental/rocketmq01/rocketmq-nameserver/data/kvConfig.json #是否启用Epoll IO模型 useEpollNativeSelector=true #端口 listenPort=9876 修改日志路径日志文件路径：/neworiental/rocketmq01/rocketmq-nameserver/rocketmq471/conf/logback_namesrv.xml 使用命令批量替换： sed -i ‘s?${user.home}/logs/rocketmqlogs?/neworiental/rocketmq01/rocketmq-nameserver/logs?’ logback_namesrv.xml 打开logback_namesrv.xml，可以看到路径已全部替换： 可根据需求修改启动脚本，如：JVM配置、堆栈大小等，路径： /neworiental/rocketmq01/rocketmq-nameserver/rocketmq471/bin/mqnamesrv 启动nameserver nohup sh /neworiental/rocketmq01/rocketmq-nameserver/rocketmq471/bin/mqnamesrv -c /neworiental/rocketmq01/rocketmq-nameserver/rocketmq471/conf/nameserver.conf >/dev/null 2>&1 &根据以上步骤启动剩下两个nameserver，配置文件直接复制使用 部署broker以master-a节点为示例 修改日志路径日志配置文件路径： /neworiental/rocketmq01/rocketmq-a/rocketmq471/conf/logback_broker.xml /neworiental/rocketmq01/rocketmq-a/rocketmq471/conf/logback_namesrv.xml /neworiental/rocketmq01/rocketmq-a/rocketmq471/conf/logback_tools.xml 执行命令：（注意不同的Broker目录不一样） sed -i 's?${user.home}/logs/rocketmqlogs?/neworiental/rocketmq01/rocketmq-a/logs?' logback_broker.xml sed -i 's?${user.home}/logs/rocketmqlogs?/neworiental/rocketmq01/rocketmq-a/logs?' logback_namesrv.xml sed -i 's?${user.home}/logs/rocketmqlogs?/neworiental/rocketmq01/rocketmq-a/logs?' logback_tools.xml 修改配置文件配置文件路径： **避坑：**集群端口配置讲解：http://www.tianshouzhi.com/api/tutorials/rocketmq/417 #端口（注意：broker启动后，会占用3个端口，分别在listenPort基础上-2，+1，供内部程序使用，所以集群一定要规划好端口，避免冲突） listenPort = 10911 #slave连接端口,配置之后无效，所以不用配 #haListenPort = 20911 #RocketMQ主目录 rocketmqHome = /neworiental/rocketmq01/rocketmq-a/rocketmq471 #集群名称 brokerClusterName = Rocketmq01 #nameserver地址，多个以分号隔开 namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876 #broker名称（每组主从节点使用相同的brokerName，方便查看） brokerName = broker-a #Broker对外提供服务地址 brokerIP1 = 172.22.36.68 #供slave同步消息的地址 brokerIP2= 172.22.36.68 #broker的id,0为主,正数0为从 brokerId = 0 #数据存放的根目录 storePathRootDir = /neworiental/rocketmq01/rocketmq-a/store #commit log保存目录 storePathCommitLog = /neworiental/rocketmq01/rocketmq-a/store/commitlog #消费队列存储路径存储路径 storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-a/store/consumequeue #删除文件时间点，默认凌晨4点。24小时制，单位小时 deleteWhen = 04 #文件保存小时数 fileReservedTime = 168 #SYNC_MASTER/ASYNC_MASTER/SLAVE 节点角色 brokerRole = ASYNC_MASTER #刷盘方式 flushDiskType = ASYNC_FLUSH #是否允许Broker自动创建Topic autoCreateTopicEnable = false #是否允许 Broker 自动创建订阅组 autoCreateSubscriptionGroup = false #linux开启epoll useEpollNativeSelector = true 修改启动脚本文件路径：/neworiental/rocketmq01/rocketmq-a/rocketmq471/bin/runbroker.sh 可根据需求，修改jvm参数、GC日志目录等： 
启动Broker
nohup sh /neworiental/rocketmq01/rocketmq-a/rocketmq471/bin/mqbroker -c /neworiental/rocketmq01/rocketmq-a/rocketmq471/conf/broker.conf >/dev/null 2>&1 &
根据以上步骤，部署其余8个broker。 友情提示：对于这种中间件集群的搭建，基本就是不同的节点，要设置不同的存储目录，不同的配置文件，由于节点比较多，操作比较繁琐，很容易出现失误，所以可以先搭建好一个节点，确保可行之后，在本地模拟线上每个机器、每个节点的目录结构，把所有的节点对应的配置文件先改好了、检查好了，最后再每个节点逐个上传、启动，一定会事半功倍！ 部署完成之后，可以进入任意broker的RocketMQ二进制包的bin目录，用命令查看集群信息： sh mqadmin clusterList -n 127.0.0.1:9876 可以看到3主3从的RocketMQ集群搭建成功！ 部署Console 下载源码地址：https://github.com/apache/rocketmq-externals.git 下载后，使用IDEA打开rocketmq-console子模块 修改配置文件检查pom.xml，修改RocketMQ版本： 需在application.properties中开启用户登录 rocketmq.config.loginRequired=true：（如果不需要设置用户登录，可忽略） users.properties文件需要拷贝一份到console的data目录，方便修改，热加载：/neworiental/rocketmq01/rocketmq-console/data 使用maven命令或IDEA将console打包，生成jar文件：rocketmq-console-ng-2.0.0.jar 上传到服务器后，java -jar rocketmq-console-ng-2.0.0.jar 启动服务即可 附录所有Broker配置文件 机器1： master-a listenPort = 10911 rocketmqHome = /neworiental/rocketmq01/rocketmq-a/rocketmq471 brokerClusterName = Rocketmq01 namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876 brokerName = broker-a brokerIP1 = 172.22.36.68 brokerIP2= 172.22.36.68 brokerId = 0 storePathRootDir = /neworiental/rocketmq01/rocketmq-a/store storePathCommitLog = /neworiental/rocketmq01/rocketmq-a/store/commitlog storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-a/store/consumequeue deleteWhen = 04 fileReservedTime = 168 brokerRole = ASYNC_MASTER flushDiskType = ASYNC_FLUSH autoCreateTopicEnable = false autoCreateSubscriptionGroup = false useEpollNativeSelector = trueb-slave1： listenPort = 10915 rocketmqHome = /neworiental/rocketmq01/rocketmq-b-s1/rocketmq471 brokerClusterName = Rocketmq01 namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876 brokerName = broker-b brokerIP1 = 172.22.36.68 brokerIP2 = 172.22.36.68 brokerId = 1 storePathRootDir = /neworiental/rocketmq01/rocketmq-b-s1/store storePathCommitLog = /neworiental/rocketmq01/rocketmq-b-s1/store/commitlog storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-b-s1/store/consumequeue deleteWhen = 04 fileReservedTime = 168 brokerRole = SLAVE flushDiskType = ASYNC_FLUSH autoCreateTopicEnable = false autoCreateSubscriptionGroup = false useEpollNativeSelector = truec-slave1： listenPort = 10919 rocketmqHome = /neworiental/rocketmq01/rocketmq-c-s1/rocketmq471 brokerClusterName = Rocketmq01 namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876 brokerName = broker-c brokerIP1 = 172.22.36.68 brokerIP2 = 172.22.36.68 brokerId = 1 storePathRootDir = /neworiental/rocketmq01/rocketmq-c-s1/store storePathCommitLog = /neworiental/rocketmq01/rocketmq-c-s1/store/commitlog storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-c-s1/store/consumequeue deleteWhen = 04 fileReservedTime = 168 brokerRole = SLAVE flushDiskType = ASYNC_FLUSH autoCreateTopicEnable = false autoCreateSubscriptionGroup = false useEpollNativeSelector = true机器2： master-b： listenPort = 10911 rocketmqHome = /neworiental/rocketmq01/rocketmq-b/rocketmq471 brokerClusterName = Rocketmq01 namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876 brokerName = broker-b brokerIP1 = 172.22.36.69 brokerIP2= 172.22.36.69 brokerId = 0 storePathRootDir = /neworiental/rocketmq01/rocketmq-b/store storePathCommitLog = /neworiental/rocketmq01/rocketmq-b/store/commitlog storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-b/store/consumequeue deleteWhen = 04 fileReservedTime = 168 brokerRole = ASYNC_MASTER flushDiskType = ASYNC_FLUSH autoCreateTopicEnable = false autoCreateSubscriptionGroup = false useEpollNativeSelector = truea-slave1： listenPort = 10915 rocketmqHome = /neworiental/rocketmq01/rocketmq-a-s1/rocketmq471 brokerClusterName = Rocketmq01 namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876 brokerName = broker-a brokerIP1 = 172.22.36.69 brokerIP2 = 172.22.36.69 brokerId = 1 storePathRootDir = /neworiental/rocketmq01/rocketmq-a-s1/store storePathCommitLog = /neworiental/rocketmq01/rocketmq-a-s1/store/commitlog storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-a-s1/store/consumequeue deleteWhen = 04 fileReservedTime = 168 brokerRole = SLAVE flushDiskType = ASYNC_FLUSH autoCreateTopicEnable = false autoCreateSubscriptionGroup = false useEpollNativeSelector = truec-slave2： listenPort = 10919 rocketmqHome = /neworiental/rocketmq01/rocketmq-c-s2/rocketmq471 brokerClusterName = Rocketmq01 namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876 brokerName = broker-c brokerIP1 = 172.22.36.69 brokerIP2= 172.22.36.69 brokerId = 2 storePathRootDir = /neworiental/rocketmq01/rocketmq-c-s2/store storePathCommitLog = /neworiental/rocketmq01/rocketmq-c-s2/store/commitlog storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-c-s2/store/consumequeue deleteWhen = 04 fileReservedTime = 168 brokerRole = SLAVE flushDiskType = ASYNC_FLUSH autoCreateTopicEnable = false autoCreateSubscriptionGroup = false useEpollNativeSelector = true机器3： master-c： listenPort = 10911 rocketmqHome = /neworiental/rocketmq01/rocketmq-c/rocketmq471 brokerClusterName = Rocketmq01 namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876 brokerName = broker-c brokerIP1 = 172.22.36.70 brokerIP2= 172.22.36.70 brokerId = 0 storePathRootDir = /neworiental/rocketmq01/rocketmq-c/store storePathCommitLog = /neworiental/rocketmq01/rocketmq-c/store/commitlog storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-c/store/consumequeue deleteWhen = 04 fileReservedTime = 168 brokerRole = ASYNC_MASTER flushDiskType = ASYNC_FLUSH autoCreateTopicEnable = false autoCreateSubscriptionGroup = false useEpollNativeSelector = truea-slave2： listenPort = 10915 rocketmqHome = /neworiental/rocketmq01/rocketmq-a-s2/rocketmq471 brokerClusterName = Rocketmq01 namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876 brokerName = broker-a brokerIP1 = 172.22.36.70 brokerIP1 = 172.22.36.70 brokerId = 2 storePathRootDir = /neworiental/rocketmq01/rocketmq-a-s2/store storePathCommitLog = /neworiental/rocketmq01/rocketmq-a-s2/store/commitlog storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-a-s2/store/consumequeue deleteWhen = 04 fileReservedTime = 168 brokerRole = SLAVE flushDiskType = ASYNC_FLUSH autoCreateTopicEnable = false autoCreateSubscriptionGroup = false useEpollNativeSelector = trueb-slave2： listenPort = 10919 rocketmqHome = /neworiental/rocketmq01/rocketmq-b-s2/rocketmq471 brokerClusterName = Rocketmq01 namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876 brokerName = broker-b brokerIP1 = 172.22.36.70 brokerIP1 = 172.22.36.70 brokerId = 2 storePathRootDir = /neworiental/rocketmq01/rocketmq-b-s2/store storePathCommitLog = /neworiental/rocketmq01/rocketmq-b-s2/store/commitlog storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-b-s2/store/consumequeue deleteWhen = 04 fileReservedTime = 168 brokerRole = SLAVE flushDiskType = ASYNC_FLUSH autoCreateTopicEnable = false autoCreateSubscriptionGroup = false useEpollNativeSelector = true |
【本文地址】
今日新闻 |
推荐新闻 |